Papers with exact inference
Deep Latent Variable Models of Natural Language (D18-3)
Copied to clipboard
| Challenge: | In this tutorial, we will discuss the challenges of applying neural variational inference to NLP problems. |
| Approach: | The tutorial will cover deep latent variable models in the case where exact inference over the latent variables is tractable. |
| Outcome: | The proposed tutorial will cover deep latent variable models in the case where inference cannot be performed tractably and when it is not . |
AMR Parsing as Graph Prediction with Latent Alignment (P18-1)
Copied to clipboard
| Challenge: | Abstract meaning representations (AMRs) are sentence-level semantic representations . lack of explicit alignments between nodes in graphs and words in sentences is a challenge . |
| Approach: | They propose a neural parser which treats alignments as latent variables within a joint probabilistic model of concepts, relations and alignments. |
| Outcome: | The proposed parser achieves the best reported results on the standard benchmark (74.4% on LDC2016E25). |
Best-First Beam Search (2020.tacl-1)
Copied to clipboard
| Challenge: | Currently, beam search is the default for decoding structured predictors . however, little work has been done to speed up beam search itself . |
| Approach: | They propose a beam search algorithm that prunes the scoring function to a monotonic sequence length, which allows for safe pruning of hypotheses that cannot be in the final set of hypothecies. |
| Outcome: | The proposed method can be implemented up to 10x faster in practice. |
Improving Coverage and Runtime Complexity for Exact Inference in Non-Projective Transition-Based Dependency Parsers (N18-2)
Copied to clipboard
| Challenge: | Non-projective dependency trees account for 12.59% of all training sentences in the annotated Universal Dependencies (UD) 2.1 data. |
| Approach: | They generalize Cohen et al.'s (2011) parser to a family of non-projective transition-based dependency parsers allowing polynomial-time exact inference. |
| Outcome: | The proposed system can be extended to include a variant that reduces time complexity to O(n6), improving over the known bounds in exact inference for non-projective transition-based parsing. |
Scaling Hidden Markov Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Hidden Markov models are a fundamental tool for sequence modeling that separates the hidden state from the emission structure. |
| Approach: | They propose methods for scaling hidden Markov models to massive state spaces while maintaining efficient exact inference and effective regularization. |
| Outcome: | The proposed methods are much more accurate than previous HMMs and n-gram-based methods, making progress towards the performance of state-of-the-art NN models. |
Headed-Span-Based Projective Dependency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing based on headed spans are available. |
| Approach: | They propose a method for projective dependency parsing based on headed spans. |
| Outcome: | The proposed method achieves state-of-the-art or competitive results on PTB, CTB, and UD Dependency parsing is an important task in natural language processing. |
Unsupervised Learning of Syntactic Structure with Invertible Neural Projections (D18-1)
Copied to clipboard
| Challenge: | Unsupervised learning of syntactic structure is typically performed using generative models with discrete latent variables and multinomial parameters. |
| Approach: | They propose a generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior. |
| Outcome: | The proposed model outperforms state-of-the-art models on part-of speech (POS) induction and unsupervised dependency parsing without gold POS annotation. |
Incorporating Word Attention into Character-Based Word Segmentation (N19-1)
Copied to clipboard
Shohei Higashiyama, Masao Utiyama, Eiichiro Sumita, Masao Ideuchi, Yoshiaki Oida, Yohei Sakamoto, Isaac Okada
| Challenge: | Word segmentation models are used to minimize the effort in feature engineering. |
| Approach: | They propose a character-based model that learns the importance of multiple candidate words for a corresponding character on the basis of an attention mechanism and makes use of it for segmentation decisions. |
| Outcome: | The proposed model outperforms the state-of-the-art models on Japanese and Chinese benchmark datasets. |
Benchmarking Approximate Inference Methods for Neural Structured Prediction (N19-1)
Copied to clipboard
| Challenge: | Structured prediction models often involve complex inference problems for which finding exact solutions is intractable. |
| Approach: | They propose to perform gradient descent with respect to the output structure directly and train a neural network to perform inference. |
| Outcome: | The proposed methods achieve better speed/accuracy/search error trade-off than gradient descent while being faster than exact inference at similar accuracy levels. |
Semi-supervised Autoencoding Projective Dependency Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for semi-supervised dependency parsing use labeled data, but they require large amounts of labeles. |
| Approach: | They propose two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. |
| Outcome: | The proposed models outperform a semi-supervised model on WSJ and UD dependency parsing data sets. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |